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The use of electronic portfolios for authentic student assessment is growing rapidly (Batson, 2002). 

Creating portfolios electronically offers a number of benefits not available using traditional paper-based portfolios. 
Advantages include the portability from one application or institution to another, wider accessibility of the portfolio, 
and the reusability of artifacts in different contexts and to create multiple portfolios. 

The two broad approaches to eportfolio implementation have advantages and disadvantages. Some experts 
advocate using generic tools (e.g., word processors, HTML editors, portable document format) to create eportfolios. 
Advantages to this approach include flexibility for portfolio authors to customize their portfolio to reflect their 
individuality and portability from one system to another. Start-up cost may also be low using this approach. 
However, generic tools carry disadvantages. Authors unskilled with the tools may suffer cognitive overload, 
producing lower quality portfolio content as they struggle to use the tools. Second, it is hard to aggregate data from 
portfolios created using generic tools. Finally, securing the contents of the electronic portfolios may be difficult if 
they are to be accessible from Internet-connected computers. 

Alternatives to the traditional generic tools are specialized electronic portfolio systems. These take 
advantage of computer databases, servers, interfaces, and custom programming. Advantages of such systems include 
more powerful data aggregation to satisfy accountability mandates, simplified security procedures, increased 
opportunities to create multiple portfolios for diverse purposes, computer- mediated communication between 
portfolio authors and their advisers, and a less steep learning curve for the portfolio author. However, disadvantages 
are evident. The portfolio author’s ability for self-expression is limited. Some customized systems are expensive to 
implement and maintain. Proprietary structures and file formats may limit portfolio portability. 

Many customized eportfolio systems can be integrated with existing student information systems and 
course management systems. An advantage of tightly integrated systems is the ability to repurpose existing 
information and artifacts for use in portfolios. However, tightly coupled systems such as these require security to 
prevent unauthorized access to private personal data. 

Selecting an eportfolio system requires rigorous attention to pedagogical and technological considerations. 
Pedagogical considerations include the level of users' technical skills, user access to technology, staff development 
needs, curricular standards to be met, the level of creativity desired, access to student reflections, and data 
aggregation needs. Additional pedagogical concerns are illustrated in the top half of Figure 1. 

Technological considerations are equally complex, including budgets for hardware, software, and technical 
support, the capacity of the institution’s infrastructure, system scalability, usability issues including plug-ins and 
utilities, authentication systems, required technical standards (e.g., ODBC- and SCORM compliance) and ongoing 
system maintenance. Additional technological concerns are illustrated in the bottom half of Figure 1. 

As technological and pedagogical considerations are addressed, a set of requirements will emerge. The 
requirements and preference decisions, along with the results of usability testing, should guide selection of an 
electronic portfolio system. This paper addresses many of these decisions in detail, describes selected electronic 
portfolio systems, and identifies additional eportfolio resources. 

Electronic Portfolios: An Overview 

For many years, artists have used portfolios to showcase their work, demonstrate their skills, and find 
employment. Portfolios are now being used in a variety of fields, including, but not limited to the humanities (Reiss, 
2001), information literacy (Fourie & van Niekerk, 1999, 2001), management (Zalatan, 2001), and nursing (Lettus, 
Moessner, & Dooley, 2001). Portfolio assessment is also gaining popularity in K-12 and higher education as both 
teachers and students collect artifacts to demonstrate skills in specific areas (e.g., Shulman, 1998; Carpenter, Ray, & 
Bloom, 1995; Karoly, 1996; Lyons, 1998; Snyder, Lippincott, & Bower, 1998; Quinlan, 2002). Several definitions 
of portfolios have been offered. One of the most frequently cited is that of Paulson, Paulson, and Meyer (1991): “A 
portfolio is a purposeful collection of student work that exhibits the student’s efforts, progress, and achievements in 
one or more areas. The collection must include student participation in selecting contents... and evidence of student 
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self-reflection’' (p. 60). Similarly, Shulman (1998) of Stanford University defines a teaching portfolio as “the 
structured, documentary history of a set of coached or mentored acts of teaching, substantiated by samples of student 
portfolios, and fully realized only through reflective writing, deliberation, and conversation" (p. 37). Barrett (2001b) 
believes “an electronic portfolio is really a living history of lifelong learning.” 

This paper discusses a cross-section of issues related to portfolios, eportfolios, and eportfolio development 
and implementation. Topics include a look at the dilemmas faced by implementers, technological and pedagogical 
concerns raised in the literature, information about the portfolio development process, clarification of important 
terms, criteria for evaluating eportfolio tools, and an examination of various eportfolio tools. Important concerns, 
both technological and pedagogical are addressed and a structure for thinking about those issues is proposed. 

Finally, the paper closes with a list of useful resources and organizations for the decision-maker interested in 
locating additional information about electronic portfolios. 

Defining Electronic Portfolios 

As technology becomes increasingly available, student portfolios are moving from paper formats to 
electronic formats. Experts distinguish between various electronic portfolios and electronic portfolio 
implementations in subtly different ways. Barrett (2000) defines electronic portfolios to include “the use of 
electronic technologies that allow the developer to collect and organize artifacts in many formats (audio, video, 
graphics, and text)" (p. 15). Barrett emphasizes that “an electronic portfolio is not a haphazard collection of artifacts 
(i.e., a digital scrapbook or multimedia presentation) but rather a reflective tool that demonstrates growth over time" 
(p. 15). Barrett (2003) distinguishes between electronic portfolios and online assessment management systems. 

Many teacher education systems are implementing electronic portfolios to meet NCATE standards. These systems 
often resemble an online assessment management system, complete with numerical scoring, rubrics, and statistical 
analyses. Trent Batson (2002) uses the term eportfolio to refer to “database-driven, dynamic web sites, not static, 
HTML-driven sites" (p. 2). The online assessment management systems are typically database-driven. These 
distinctions can result in vastly different implementations. 

The use of eportfolios is growing in popularity. Electronic portfolios are used for such diverse purposes as 
assessment, accreditation, reflection, and professional development. An increasing number of eportfolio developers 
are entering the market, facilitating the portability of file types including text, graphics, video, audio, photos, and 
animations. This increasing popularity is due, at least in part, to the increasing use of digital formats for student 
work and the need to organize, search, document, and transport this work. These activities take advantage of 
computer capabilities of storage, display, retrieval, and communication to change curricula (Batson, 2002). 

The American Association for Higher Education (2003) has proposed a “Taxonomy for Electronic Portfolios” to 
facilitate discussions about the various kinds of eportfolio implementations. This taxonomy has three main 
discriminators: context, author, and purpose. Understanding these dimensions for your specific need is important to 
select the “right" portfolio system or tool. The context describes the setting in which the portfolio was developed. 
Contexts include courses, programs, institution, inter-institutional, and independent portfolios. For example, 
depending on the context, data aggregation may or may not be an important factor to consider. The author of an 
electronic portfolio may be a student, faculty member, administrator, organization, or other individual including 
those not specifically affiliated with the learning organization. Knowing the portfolio authors, and their skills, 
should influence your decision. Purposes for portfolios include development (self-assessment, advising, 
documenting learning over time, documenting professional development, building the curriculum, adding to the 
knowledge base of or among the disciplines), evaluation (achievement of learning outcomes, high stakes evaluation, 
accreditation, promotion and tenure), and presentation (showcasing achievement, publicizing organizational 
reflection and progress, and responsiveness to state and national need for information). The variety of purposes will 
determine the level of creativity and flexibility required of the selected system. 

Dilemmas Eportfolio Authors Encounter 

Electronic portfolios can be developed using one of two broad approaches: using generic tools (word 
processors, HTML editors, portable document format, authoring software) or using customized information systems 
including databases, servers, interfaces, and custom programming (Gibson & Barrett, 2002). Each approach has both 
advantages and disadvantages, forcing eportfolio authors to make choices that are oftentimes difficult. 

Joanne Carney (2002) completed case study research that examined the decision-making of electronic 
portfolio authors. Her findings help illuminate the decision about which type of electronic portfolio tool to 
implement. She identified six dilemmas that must be confronted by those interested in electronic portfolio 
implementation. Those dilemmas are: 
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Multiple-purpose dilemma - Students are encouraged to create portfolios for multiple purposes and 
multiple audiences. Too often, these purposes are in conflict with each other and the result is that none of the 
purposes are served well. An example of this type of conflict can be found in pre -service teacher education. Pre- 
service teachers are asked to include reflections on their teaching internship in their portfolio. They are encouraged 
to reflect on their mistakes and share what they learned from that mistake. However, teacher educators also suggest 
the portfolio be used for credentialing purposes. Students may be understandably reluctant to share the same 
information about their mistakes with a licensing agency that they share with a trusted professor. This conception of 
the portfolio also being used in a high-stakes assessment situation often motivates the portfolio author to downplay 
any problems they had in teaching despite the learning benefit from a critical examination of such a problem. 
Portfolio authors are likely to experience a constant tension between the conflicting purposes of their portfolio 

Personal -rev elation dilemma - Electronic portfolios are frequently placed on the Internet for a worldwide 
audience. Revealing personal difficulties is a source of discomfort for many people and can be used against them in 
countless ways. 

Cognitive-overload dilemma - This dilemma is exacerbated by the use of generic tools. Portfolio authors 
lacking the technological skills to produce an electronic portfolio spend cognitive capital coping with their 
technological deficiencies at the expense of portfolio content. Even portfolio authors with a high degree of skill 
spend capital planning and executing specific tasks and exploring new capabilities. With customized systems, the 
cognitive overhead is less than that required to use generic tools. 

Self-expression dilemma - Electronic portfolio authors using customized systems are often constrained by 
the systems they use. These authors exert less control over format, appearance, and structure than a portfolio author 
skilled in using generic tools. Portfolios are often considered a very personal reflection of self, and this constraint 
can be a source of dissatisfaction. On the other hand, the unskilled users of this system can readily select an 
aesthetically -pleasing, but pre -determined template to create an eportfolio. 

Dead-end dilemma - One of the original purposes of portfolios was to serve as a map or plan for future 
growth and development. Authors of paper portfolios often consider their portfolio to be “done” while electronic 
portfolio authors are more likely to consider their portfolios “a work in progress.” Authors using generic tools have 
portable files that can be taken from school to school or into the workplace. Customized systems using proprietary 
file formats may cause users to hit a dead-end when they attempt to move their files or teacher candidates to 
abandon their portfolio upon graduation or the next bill to renew the service. 

Data-aggregation dilemma - Customized portfolio systems are specifically designed to aggregate data for 
such purposes as program evaluation and accreditation. With that as a goal, institutional requirements may 
discourage portfolio authors from documenting deficiencies, despite the potential learning opportunities from a 
closer examination of such difficulties. 

Advantages and Challenges in ePortfolio Implementation 

The use of electronic portfolios offers a number of advantages over traditional paper-based portfolios, such 
as portability, accessibility, distribution ability, and repeatability of performances (Sheingold, cited in Barrett, 1998). 
With portfolios, students participate in their own assessment, rather than rely solely on a teacher’s evaluation of their 
work(Mullin, 1998). 

The ePortConsortium White Paper (2003) contends that the rise in e-learning has contributed substantially 
to the viability of electronic portfolio adoption. At institutions where elearning is in place, there is an existing 
network that can also be used for an electronic portfolio initiative. Many elearning systems use database-driven 
course management systems. Oftentimes, these systems capture information that can be repurposed for an electronic 
portfolio program. Furthermore, many of the artifacts a portfolio author may decide to use are already available in 
electronic formats and easily transferred to an eportfolio system. Additionally, the data aggregation capabilities 
already present in elearning databases and electronic portfolio systems can facilitate working with accountability 
mandates existing in education today. 

Well-designed electronic portfolio systems facilitate ongoing dialogue between portfolio authors and their 
audience members. Annotation capabilities and threaded discussions allow the preservation of such dialogues for 
documentation of learning and evidence included in reflections. Electronic portfolios take advantage of hyperlinking 
to provide alternative routes through an individual portfolio. Furthermore, authentication and security practices 
allow an eportfolio author to determine who is able to see specific artifacts and who is denied this access. 

Finally, electronic portfolios offer opportunities in the area of career development. Employees and 
eportfolio authors can maintain and update their personal information to showcase skills, certifications, 
accomplishments, and awards (ePortConsortium, 2003). However, issues of verification must be addressed (Batson, 
2002). There are foreseeable circumstances in which a portfolio reviewer, be it a potential employer or graduate 
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school admissions counselor, would want to ensure work represented in the portfolio was actually that of the 
portfolio author. 

Despite the numerous benefits afforded by electronic portfolios, eportfolio innovators must address a 
number of challenges. The authors of the ePortConsortium White Paper (2003) highlight seven areas in which 
challenges may arise: hardware and software, security and privacy, intellectual property and digital rights, usability, 
assessment, acceptance and managing expectations, and long-term maintenance. 

1. Hardware and software challenges include the decision to use a generic tool or customized portfolio 
system as well as identifying funding for long-term licensing of such products. Further considerations include 
interoperability with existing systems such as student information systems and human resources system. Systems 
should be scalable to support further growth of the institution. Levels of expertise residing in the organization must 
be considered to determine development and deployment options. 

2. Security and privacy issues must be addressed to determine who will access portfolio content and system 
data. Policies to address such concerns must be specified before extending access to external audiences such as 
potential employers and program evaluators. The institution’s responsibilities under the Federal Educational Rights 
and Privacy Act (FERPA) must also be considered. 

3. Intellectual property and digital rights concerns are growing every day. Issues of ownership of materials 
uploaded to electronic portfolios must be addressed and methods to resolve disputes put in place. Mechanisms for 
verifying and preserving records of authorship should be implemented. “Fair use” guidelines must be made clear. 

4. Usability issues to address include identifying the plug-ins and utilities required to use the eportfolio 
system, the file formats that will be supported, browsers and computer operating systems that can be used, and the 
bandwidth and other technologies required to create and maintain an electronic portfolio. Additionally, the structure, 
navigation, and system performance of the eportfolio system will enhance or detract from usability. 

5. Assessment challenges include decisions on individual, course, program, and institutional assessment. 
Curricular and performance standards appropriate for such assessments must be identified and decisions about how 
assessment data is maintained must be made. Finally, decisions about who will have access to assessment data must 
be considered. 

6. Acceptance and managing expectations is a challenge with any large-scale implementation. Managing 
user expectations and fostering acceptance among users are critical tasks to ensure long-term success. Clarifying 
expectations and responding to suggestions from a diverse user community including alumni, staff, students, 
evaluators, and administrators must be addressed. 

7. Long-term maintenance issues include an author’s ongoing access to his or her eportfolios. Decisions 
about how and when to delete files or move them to long-term storage must be addressed. Garbage collections 
routines to delete orphan pages and dead links should be examined. 

Important Concerns to Address 

Implemented of electronic portfolio programs must address a number of issues, including both 
technological and pedagogical concerns. 

Technological Concerns 

Batson (2002) has identified four areas of concern: storage, security, certification, and industry stability. 

1. Storage - An organization implementing an eportfolio program must consider how work will be stored 
and for how long. Furthermore, multimedia and graphics files can be quite large, requiring large quantities of costly 
storage space. In addition, accessing older storage media is problematic. For example, machines capable of reading 
data stored on a 5.25” disk are rare today. 

2. Security - Maintaining the security of personal information is an important consideration among 
information technologists today. Electronic portfolios may be containers for private information including social 
security numbers, grades, and intellectual property. A security breach could potentially expose the user to identity 
theft and an organization to a lawsuit. 

3. Certification - If an educational organization intends to certify work contained in electronic portfolios is 
authentic, a system needs to be created enabling faculty to prevent further changes to certain artifacts and data. 
Authenticity is critical for effective assessment, evaluation, and credentialing. 

4. Industry stability - Many companies offering electronic portfolio programs are relatively new and often 
small. Educational institutions making a long-term commitment to an electronic portfolio program will face tough 
decisions relying on new small vendors with an uncertain future. 

The ePortConsortium’ s White Paper (2003) addressed additional concerns including system infrastructure, 
interoperability, and technological standards. System infrastructure concerns examine the relationships between 
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eportfolio systems and other campus systems already in place. A single department eportfolio system, such as 
University of Central Florida is undertaking Fall 2004, can be a standalone implementation with little to no impact 
on existing system infrastructure. While integration of an eportfolio system with a Student Information System (SIS) 
is achievable, it is not necessary. One advantage of a stand alone implementation is rapid deployment, however a 
disadvantage can be duplication of effort and content across various campus systems. Other deployment options 
include integrating the eportfolio system with a course management system or full integration with campus 
enterprise systems. An advantage to these types of deployment is a single authentication system for all campus 
systems. Specific security and certification issues can be addressed more broadly in this type of implementation. 
Additionally, such systems may be tightly integrated with course management systems, learning systems, grants 
management systems, and other campus wide systems. 

Interoperability and technological standards considerations can insure eportfolio systems are able to 
communicate with other campus systems to receive and pass information needed by the various systems. 

Information requests may include access to data created by and about users, standardized data structures, and 
verification and digital rights management. Interoperability will also assist in managing workflow between various 
constituents. Adoption of technological standards can allow content to be reused, integrate data from one system 
into other systems, and facilitate authentication and authorization. 

Usability and accessibility by “special needs” populations must also be considered. Adherence to specific 
document standards (i.e. PDF, jpg, etc.) can help insure archived documents can be read in the future. Investigation 
into the National Archives and Records Administration’s (NARA) PDF/A standard or some other long-term archival 
format is important. Widespread adoption of carefully defined technological standards can facilitate portability of 
eportfolios and data from one institution or one system to another. This is essential for the long-term usefulness, 
accessibility, and viability of electronic portfolios systems. 

Pedagogical Concerns 

Issues of validity and reliability appear repeatedly in examinations of large-scale portfolio assessment 
deployments (Koretz, 1998; Murphy, Bergamini, & Rooney, 1997; Stecher, 1998). Recently, Wilkerson and Lang 
(2003) reported on the legal and psychometric issues surrounding the use of portfolios for high-stakes assessment 
purposes such as teacher certification assessment. They raised important concerns about the psychometric properties 
of validity, reliability, absence of bias, and fairness. 

Gearhart and Herman (1998) examined the social context of portfolio development. They found wide 
variability in how portfolios are implemented and the time teachers provide for students to receive assistance with 
revisions and choosing artifacts. Though such concerns are less critical for classroom assessments where a teacher 
understands the context in which the portfolio was constructed, this social milieu introduces reliability problems for 
large-scale assessments. Heller, Sheingold, and Myford (1998) examined portfolio raters and their reasoning while 
conducting portfolio assessment. They were specifically interested in determining when problems with the rating 
process introduced threats to validity. They noted raters found portfolios with uneven evidence were most difficult 
to rate. 

In addition to the technological concerns of storage, security, and portability, Barrett (2002) added the 
pedagogical concerns of reflection, publishing, and linking and grouping to the list issues in implementing electronic 
portfolios. She stressed the importance of reflection to provide portfolio authors an opportunity to demonstrate 
achievements and to set future learning goals and direction. Under the category of publishing, she raised issues such 
as the ability to individualize portfolio presentations to avoid “cookie cutter” effects and facilitate creativity. Further, 
she endorsed the ability to create a variety of portfolios for different purposes. Types of portfolios might include a 
learning portfolio focusing on the learner as the audience, a highly structured assessment portfolio to demonstrate 
achievement and receive faculty feedback, employment portfolios, and showcase portfolios demonstrating growth or 
highlighting specific achievements. 

Work by Wolfe and Miller (1997) indicated teacher time and perceived difficulty in scoring are the biggest 
barriers to portfolio assessment and such concerns must be carefully addressed. They also noted perceived barriers 
decreased as teachers gained more experience with portfolios. 

Kathleen Yancey (2001) wrote about the issues to be addressed in the portfolio planning process and has 
identified six critical issues, both technological and pedagogical. These included storage, leveraging the potential of 
the electronic environment, defining required technical skills for both students and faculty, determining the level of 
creativity to be supported (i.e. templates or individual creations), determining a schedule for faculty review and 
feedback, and defining the life of the portfolio. 
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Developing Portfolios 

To understand the technological and pedagogical considerations associated with developing electronic 
portfolios, interested parties can review work from the traditional paper portfolio movement. Burke, Fogarty, and 
Belgrad (1994) have elaborated on a 10-phase portfolio development process (see Figure 1). Briefly, those phases 
involve: 

• Project the purposes and uses of the portfolio to be developed. 

• Collect and organize the artifacts to include in the portfolio. A variety of organizational schemes 
and devices are appropriate. 

• Select the artifacts to be included. Selections may be based on general guidelines. 

• Interject the personality of the portfolio author by color selections, choices regarding page layout, 
or setting the mood or tone. 

• Reflect metacognitively. This involves descriptions of each artifact to help the portfolio reader 
understand the importance of the selected artifact and how that artifact fits into the overall 
portfolio and portfolio goals. 

• Inspect and self-asses goals. This is an examination of the overall portfolio to determine what, if 
any, material must be added or re -aligned. 

• Perfect, evaluate, and grade (if you must). In this stage, portfolio authors add the finishing touches 
and prepare the portfolio for the next stage. Teachers may choose to grade a portfolio at this stage. 

• Connect and conference. Considered by Burke, Fogarty, and Belgrad to be a critical stage in the 
assessment process, the portfolio conference allows portfolio authors to share their work with 
parents and receive critiques from others. 

• Inject and eject. It is in this stage that the portfolio author performs ongoing maintenance and 
updating of his or her portfolio. New materials are added and obsolete materials are removed. 

• Respect accomplishments and show with pride. In this ongoing stage, portfolio authors can show 
their portfolios to others. Such exhibitions offer insight about the portfolio author to portfolio 
viewers. 

Barrett (200 la) views “portfolios as a process rather than a product - a concrete representation of critical 
thinking, reflection used to set goals” (p. 1). She situates her discussion of developing electronic portfolios in two 
bodies of literature - that of portfolio development and of multimedia development. Barrett (2000) developed a five- 
stage model of electronic portfolio development based on these discussions. 

The first stage defining the portfolio context and goals. In this stage, the portfolio author must identify the 
purpose and audience for the portfolio, the goals or performance standards that will provide the organizing 
framework for the portfolio, and identify the software to develop the portfolio. 

The second stage is a working portfolio. In this stage, the portfolio author uses the selected software to 
collect digital artifacts representing achievements and efforts. The third stage is the reflective portfolio in which the 
learner chooses specific artifacts to be included in the portfolio and completes reflections. Care must be taken to 
protect the privacy of these reflections as appropriate. 

The fourth stage, the connected portfolio, is unique to the electronic portfolios in that it allows the portfolio 
creator to insert hyperlinks to various documents and artifacts to create additional meaning. Documents are 
hyperlinked in a way that allows navigation. All artifacts have been inserted into the document and the portfolio is 
ready to present to others. The fifth and final stage is the presentation portfolio. In this stage, the portfolio is shared 
with appropriate audiences in a medium of the learner’s choosing. Some are published on the Internet, CD, or video. 
At this stage, some advocate using student -led conferences or interviews to present the portfolio to the appropriate 
audience (Lyons, 1998; Shulman, 1998; Snyder, Lippincott, & Bower, 1998). 

Developing Eportfolio Tools 

Today’s discussions about eportfolio tools involve the technical community and the academic community. 
At times, commonly used terms have different meanings in the different communities. Usage of these terms appears 
to be imprecise and these authors believe discourse can be enhanced by a clarification of the terms. 

Standards vs. Requirements A number of organizations are currently trying to define standards and 
requirements for electronic portfolio systems. The term “requirements” is ambiguous. Members of the instructional 
community speak of requiring a certain number of artifacts, or a certain type of artifact to provide evidence of 
specific competencies. In the technical community, the term “requirements” refers to a list of things adopters want 
the system to be able to do. 
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The term “standards” also has multiple meanings. In the context of instruction, “standards” are typically 
promulgated by an external organization such as an accrediting body, a department of education, or professional 
organization. In the technical community, a “standard” means a rule or description to which other things must 
conform. Examples include units of measurement (inch, pound, or hour) or specific protocols (Ethernet or VHS). 

Such technical standards are typically owned, controlled or defined by a specific organization. For 
example, the Institute of Electrical and Electronics Engineers (IEEE) defines many of the standards associated with 
computer networks. Product manufacturers adhere to these standards to ensure Toshiba, Dell, IBM, and Apple 
computers can share data on the same network. 

Similarly, networking companies adhere to standards to allow various products to interoperate. For 
example, IEEE’s 802.11 standard specifies requirements to insure the interoperability of wireless networking. As a 
result, wireless access points and wireless network cards manufactured by a number of different companies can all 
work on the same network. The first author’s Apple AirPort card, her professor’s Cisco wireless networking card, 
and her friend’s Netgear wireless networking card all allow wireless access to the UCF wireless network. The brand 
of the university’s wireless access point is unknown and unimportant from a user perspective. The products all work 
together (a user requirement) due to meeting certain specifications (a standard). 

Open Source and Open Standards Open source is increasingly visible in today’s higher education 
institution. The open source software movement has provided successful software projects including Linux, Apache, 
OpenOffice, and Sakai. The open source software movement is based on the wide distribution of source code, 
enabling any programmer to make improvements and modifications for further distribution. Such software is 
typically provided at no cost, or for nominal fees. However, this is not always how the term “open source” is used. 
Some proprietary vendors make their source code available to customers for increased customization and improved 
interoperability. It is still considered by many to be open source software because the source code is open. 

Open standards, such as Shareable Content Object Reference Model (SCORM) are generally driven by 
organizations attempting to define protocols to increase interoperability of systems. Such organizations include IMS 
Global Learning Consortium, Advanced Distributed Learning (ADL) network, and the IEEE Learning Technology 
Standards Committee. 

Over a period of six years, the University of Minnesota developed an electronic portfolio system for use by 
students, faculty, and staff. Recently, the code for this system was donated to the Open Source Portfolio Initiative 
(OSPI), a collaborative effort to create an electronic portfolio system by the open source software community 
(Treuer & Jenson, 2003). According to the Open Source Portfolio v. 2 Concept Paper (OSPI, 2003), efforts to 
further develop this code place interoperability and adherence to industry standards as a high priority. Similarly, the 
ePortConsortium (2003) advocates for adherence to open standards. 

As electronic portfolio adopters begin to consider the eportfolio tools they will use, a decision about open 
source and open standards is necessary. James Dalziel (2003) examines these two movements and the interactions 
between them. The decision is not an easy one, involving tradeoffs that must be considered carefully. 

Evaluating ePortfolio Tools 

The discussion in this paper thus far has focused on describing and defining terms associated with 
electronic portfolios. With the foundation for decision-making laid, the remainder of the paper addresses how an 
eportfolio implementer can evaluate and select eportfolio tools. 

Eportfolio adopters must carefully examine their context and requirements to develop criteria against which 
to evaluate eportfolio systems. Gibson and Barrett (2002) offer 10 broad criteria to consider when comparing the 
generic tools approach to the customized system approach. Those 10 criteria are: 

1. Planning and goal setting - A portfolio can be used as a tool for portfolio authors and advisers to set 
future goals and plan further learning experiences. In a customized system, this process may be prompted by the 
system and facilitated by documentation of conversations and linking to potential goals and standards. 

2. Framework for creativity - A portfolio reflects the creativity and personality of the author. 

Considerations under this category include the electronic portfolio author’s ability to individualize the portfolio 
using the tools at hand. A portfolio author using generic tools is only limited by the author’s skill with the tool. 
Customized systems may offer few, if any, choices for structure and appearance. 

3. Communications - Planning, developing, and publishing an electronic portfolio requires communication 
between the portfolio author and a mentor or adviser. An electronic portfolio system should be evaluated for 
capabilities to support this ongoing dialog. 

4. Collaboration tools - Examples of collaborative tools include threaded discussions and whiteboards. 
Electronic portfolio systems support collaboration to varying degrees. 
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5. Reflective process - An electronic portfolio should support learner reflections. Multiple audiences need 
to be accommodated which will likely involve addressing security concerns. 

6. Connection capabilities - An electronic portfolio must have the capability to link various work products 
and reflections. The system should support linkages for multiple purposes and multiple audiences. 

7. Organizational flexibility - Electronic portfolios should be organized in flexible ways to accommodate 
multiple views, purposes, and audiences. 

8. Display flexibility and transportability - Portfolio authors should be able to display their electronic 
portfolios in various ways. Proprietary display systems will limit this flexibility. 

9. Data and information - Data and information may be aggregated to support program evaluation, 
accreditation, and curricular decision-making. Data collection efforts must consider privacy and confidentiality 
concerns of electronic portfolio authors. 

10. Start-up costs and maintenance - Start-up costs and long-term maintenance needs must be calculated to 
determine if the portfolio system is cost-effective. 

Additionally, eportfolio adopters will wish to review a variety of requirements and standards proposed by 
various groups. Adherence to certain technical standards may form the basis of some requirements against which an 
electronic portfolio system must be evaluated prior to adoption. Lists of such standards can be found in Treuer and 
Jenson (2003), as well as the ePortConsortium White Paper (2003). 

Consumer’s Guide to Selected Electronic Portfolio Systems 

To analyze four e-Portfolio systems (or assessment management systems), from a consumer’s perspective, 
the authors used the system based on several assumptions about the institution environment and institutional goals, 
the system users, and the targeted audiences. The eportfolio systems selected for further investigation were LiveText, 
TaskStream , The OSPI, and Chalk and Wire. 

Assumptions about institutional environment and goals 

Implementation of the eportfolio system would take place in a higher education institution with at least 
bachelor and masters programs in multiple disciplines. The authors assumed the organization required an electronic 
portfolio system capable of integrating with existing student and course management systems, hardware, and 
software, as well as meeting the diverse present and future needs of faculty, students, and the community. However, 
the authors did not review interoperability with other systems. 

The authors assumed the institution provided on-campus computer access and a computer network to 
faculty and students. The computer hardware specifications assumed at least 256 megabytes of RAM, a 20 gigabyte 
hard drive, a microphone and speakers, a scanner, a read/write CD or DVD drive, a floppy drive, a high-speed USB 
port, and Internet access. Installed computer software was assumed to include an office suite application (i.e., word 
processor, spreadsheet, and presentation tools), Internet browser, Adobe Acrobat reader, and other freeware 
necessary to view some Internet documents and Web sites. Minimal graphic software tools (i.e., paint, Netscape 
Communicator, etc.) were also assumed to be available. 

The institution’s document management and storage systems assumed access through Internet or network 
to a central server, either located at the institution or through the service provider, and the e-portfolio system would 
need to provide a method for organizing and maintaining digital artifacts. 

Assumptions about system users 

For this evaluation, the authors envisioned two primary audiences: instructors and students. To conduct the 
evaluation, we assumed both instructors and students were proficient in performing basic tasks in word processing 
and spreadsheet software, e-mail, and Internet browsing and therefore capable of entering data into a pre -designed 
form and uploading various document formats (e.g., doc, xls, pdf, ppt, jpg, etc.) to create an artifact database. 

From the instructor’s perspective, the purpose of the e-Portfolio is to demonstrate the student’s learning and 
growth as he/she progresses through the program. As an assessment tool, the eportfolio also provides physical 
evidence of knowledge and skills gained, as well as reflections of the learning experience and interpretation of 
learning and skill development. The instructor requires an eportfolio system providing the student flexibility to 
creatively demonstrate accomplishments, as well as development of evaluation rubrics based on program standards 
and generation of both individual and aggregated assessment data for student achievement of standards reports. 
Instructors also desire the option of developing and presenting a collection of exemplary portfolio artifacts for 
illustrative purposes. 
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Although the student may recognize the value of the instructor’s application of e-Portfolios in learning, 
his/her perspective also focuses on future employment application. The student requires an e-portfolio system 
providing flexibility and creativity to effectively market him/herself. Students need to be able to publish the 
portfolios for presentation in a variety of formats: hard copy, videotape, DVD, CD-ROM, and Internet. 

In terms of context, the authors assumed the eportfolio system would typically be used as part of classroom 
instruction. As such, professors and instructors provide criteria and standards to use when selecting artifacts to 
include in the portfolio, along with a rubric explaining how the portfolios would be evaluated. The authors assumed 
the e-portfolio system would need to have both public and private student self-reflection areas, including 
documentation of future learning goals. Likewise, a need for the system to provide for confidential feedback from 
instructors was an assumed functionality. 

Assumpti ons about portfolio audiences 

Generally speaking, the authors assumed the audience to receive a complete eportfolio would be 
individuals from a higher education institution, instructors, students, their families, and potential employers. At a 
minimum, all members of the targeted audiences had access to a computer with Internet access or a CD or DVD 
player. Software requirements included an office application suite, Adobe Reader, and a web browser. 

Although cost was a primary factor in the institution’s decision criteria, faculty and student usability was 
more critical. The criteria used to evaluate e-portfolio system usability originated from work by Gibson and Barrett 
and included: 

• Planning and goal setting: Students’ portfolios can easily be evaluated against a variety of standards 
templates. 

• Creativity ability: Students can creatively build and edit portfolios to best illustrate their individuality. 

• Facilitated communications between instructor and student: System can provide internal message 
system, as well as options for instructors and peers to comment on each other’s work. 

• Collaboration tools: System can provide collaborative learning and online collaboration opportunities 
among peer and support circles. 

• Support for reflective processes: System offe rs areas for students to communicate reflective aspects of 
projects, assignments, and courses. 

• Connection and linking capabilities: System can provide methods for linking assignments, reflections, 
and other creative work products to tailor the portfolio for different audiences. 

• Organizational flexibility: System can provide opportunities for reframing portfolios for different 
applications and situations. 

• Display flexibility and transportability: Eportfolio can transfer to other formats and platforms. 

• Data and information: System can provide a variety of customizable and formatted reports generated 
by authorized users. 

• Start-up costs and maintenance: System start-up costs and ongoing maintenance are minimal due to 
ease of system administration. System administration can provide ability to associate users and 
courses; allow users to set-up and edit course profiles; offer a robust, useful, and accurate help system, 
providing clear directions regarding system features and use; enable departments to customize the 
system for unique purposes. 

• The authors of this paper added system functionality and usability: Students can log in and out of the 
system easily; determine which system’s options are available for their use; select user profiles and 
types; easily create, organize, and upload work online; submit assignments online. Instructors can 
easily create assignments online, providing corresponding instructions and assignment details. System 
security features are available and easy to apply by both instructors and students. 
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LiveText 



Criteria 


Results 


Planning and goal setting 


Instructors have convenient access to a wide variety of standards which can be used 
to create individual or institution-wide rubrics for evaluating student portfolios. 


Creativity ability 


Limited creativity is supported with the ability to upload and incorporate images. 
Images can be placed above or below text, or to the left or right of text, depending 
on the user’s wishes. The Java editor offers user control on heading type, a limited 
font selection, styles (bold, italic, underline), a small color palette, and access to 
bulleted and numbered lists. Six site theme colors are available: sage, lilac, classic 
blue, cranberry, grape, and sunset. The selected theme is applied across the entire 
site, not just one portfolio. 


Facilitated communications 
between instructor and 
student 


Portfolios can be shared between instructors and students. The instructor receives 
portfolios for review in a section of “My Desk”. Rubrics to evaluate specific 
documents or portfolios in general are available at the institutional level or can be 
created by individual instructors. These rubrics offer commenting capabilities. 
Users can enable discussion threads on each page. 


Collaboration tools 


The user can enable discussion threads on each page. Students can share their 
portfolios and documents with instructors, peers, and visitors. Commenting is 
supported. 


Support for reflective 
processes 


Institutions can create sample reflections or instructions for reflections within 
templates. These can be modified by the user for individual use. Artifacts can be 
attached or linked to the reflections. 


Connection and linking 
capabilities 


Linking capabilities are good, supporting links to documents both internal and 
external to the system. LiveText also supports attachments. 


Organizational flexibility 


Permits creation of courses, lesson plans, portfolios, and projects; User can create 
multiple instances of each project type and link from one component to another. 
Supports multiple portfolios with user-defines sections in each. Also allows 
institutions to create templates for users to modify as needed. 


Display flexibility and 
transportability 


Offers option to print, permitting selection and preview of print job. Also exports 
files to a zipped html archive of documents, images, and attachments. LiveText 
branding is removed and users can work with CSS to apply a new look to their 
work. 


Data and information 


LiveText is strong in this area, collecting and aggregating data across professors for 
use in the accreditation process. 


Start-up costs and 
maintenance 


Start-up cost is $79 or $99 per student, depending on package selected; 
maintenance $5/month after graduation. Cost is shifted to the student. 


System functionality and 
usability 


Login and logout is easy. Availability of options is readily apparent. Uploading 
work is easy. Sharing work with others is not difficult; the default security settings 
protect individual work. The system is complex enough to be daunting to the 
novice user. Highly proficient users should be able to figure out the system fairly 
easily. On the Macintosh platform, the biggest usability problem is “bugginess” 
with at least one bug resulting in lost and irretrievable data. 



Bugs encountered: 

Loading Preview - While creating a new portfolio and choosing a template, the first author saw a message 
LiveText was loading a preview. The preview never loaded. The author attempted loading twice using IE 5.2 on 
Mac OS 10.3.5 with 640 MB of RAM. 

Java Editor - Scrolling through long documents resulted in erratic behavior, making it difficult to select 
large blocks of text (for example, to change fonts). Apostrophes and quotation marks regularly turned into question 
marks, probably due to copy and paste procedures. 
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TaskStream 



Another e-Portfolio system evaluated based upon the above criteria was TaskStream 
( http://www.taskstream.com) . The opening screen for TaskStream provides a logical, organized view of the system 
tools and features, making it extremely user friendly. 



Criteria 


Results 


Planning and goal setting 


TaskStream offers a wide variety of standards for aligning student portfolios with 
the curriculum, as well as tracking standards and generating reports. 


Creativity ability 


Due to the templates offered through TaskStream, students have limited creativity 
options for displaying and demonstrating work. 


Facilitated communications 
between instructor and 
student 


TaskStream offers an internal messaging system for direct communication 
between the instructor and students. New messages are indicated by a blinking 
“New Message” icon in the upper right corner of the screen. 


Collaboration tools 


TaskStream allows students to share their e-Portfolios with other students, both 
within the system, as well as external e-mail. Based upon the level of access 
provided, students could collaborate to create a portfolio assignment for a given 
class. 


Support for reflective 
processes 


TaskStream provides a text box for student reflection at the time the student is 
uploading the artifact. Students can upload additional documents to support their 
reflective comments. 


Connection and linking 
capabilities 


Although TaskStream offers opportunities for providing links to other online 
work, as well as uploading work, limited interaction can be created between and 
among work products. 


Organizational flexibility 


TaskStream appears to offer some flexibility for reframing portfolios for a variety 
of applications (e.g., school, job search, etc.) through the “Publish/Share” option. 


Display flexibility and 
transportability 


Student portfolios are created and maintained on the TaskStream system. 
Although the uploads and links could be transported to other e-Portfolio systems, 
students would basically be re-creating their portfolios on the new system. 


Data and information 


TaskStream offers a variety of formatted, printable reports. 


Start-up costs and 
maintenance 


TaskStream provides system administration options to both instructors and 
students. Students can associate, set-up, and edit course profiles. The help system 
is useful and comprehensive, providing easy to follow directions regarding system 
features and use. System customization by different departments for unique 
purposes is possible based on standards and template options 


System functionality and 
usability 


TaskStream is relatively easy to access (logon) and provides a comprehensive 
view of system options on the entry screen. Instructors and students can easily 
determine and select user profiles and types, as well as system security features. 
Both instructors and students can easily create, organize, and upload assignments 
and work online, as well as provide corresponding instructions and assignment 
details. 



Open Source Portfolio Initiative (OSPI) 



The OSPI follows a three-step process (enter, share, view) for portfolio development. This simple sequence 
makes the system useable, but many will find themselves constrained by inflexible organizational capabilities. 



Criteria 


Results 


Planning and goal setting 


Not available 


Creativity ability 


There is little flexibility permitted for individuals. The default version 1.5 only 
permits individuals to show or not show a specific element. There is one template 
available. 


Facilitated communications 
between instructor and 
student 


Commenting features are enabled by default, offering opportunities for students 
and peers to comment on work. The notification system doesn’t work - at least 
not on the Macintosh using Mozilla or IE. 


Collaboration tools 


The only collaboration tools available are the commenting tools. 


Support for reflective 
processes 


Support for reflection is adequate. With customization by the institution, models 
of reflection can probably be provided and edited by individuals. 


Connection and linking 


Connection and linking capabilities are “buggy.” The material manager permits 
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capabilities 


users to upload and organize a variety of file types and URLs. These can then be 
attached to various portfolio elements. However, viewing these artifacts in 
preview mode generates error messages in both IE and Mozilla on a Macintosh. 


Organizational flexibility 


The default installation of the OSPI offers no flexibility for individual users. 
Because the system is Open Source, institutions with sufficient technical expertise 
on staff can substantially customize the system, but these customizations are 
unlikely to result in a highly customizable system for the end user. 


Display flexibility and 
transportability 


The system purports to offer the capability of downloading user portfolios. The 
first author couldn’t make this work on her Macintosh. IE wouldn’t allow me to 
download. Mozilla allowed the download, but then the resulting zip file would not 
extract. 


Data and information 


Not available. 


Start-up costs and 
maintenance 


Start-up and maintenance costs will vary widely depending on an institution’s 
implementation of the portfolio initiative and the in-house expertise for 
maintenance and configuration. Specific hardware requirements will need to be 
satisfied. 


System functionality and 
usability 


Adding an attachment in the personal information section caused the Apache 
server to generate an exception report in both Mozilla and IE. 



178 

















Chalk and Wire 



One of the e-Portfolio system’s evaluated based upon the above criteria was Chalk and Wire 
( www.avenet.neth Although Chalk and Wire has a relatively user friendly e-Portfolio system, some aspects of the 
system were not effective. 



Criteria 


Results 


Planning and goal setting 


Due to the structure of the Chalk and Wire system, students’ portfolios could be 
easily evaluated based upon the desired standards. 


Creativity ability 


Chalk and Wire did not provide very much flexibility for student creativity. 
Students can select templates based on a few standard system templates or upload 
their own. Students can select photographs and illustrations from a few system 
standards or upload their own. Students are unable to move text boxes or display 
assignments or work products except in the format prescribed by the system. 


Facilitated communications 
between instructor and 
student 


Chalk and Wire does offer an internal messaging system for direct communication 
between the instructor and students. 


Collaboration tools 


Chalk and Wire does allow students to share their e-Portfolios with other students. 
Based upon the level of access provided, students could collaborate to create a 
portfolio assignment for a given class. 


Support for reflective 
processes 


Chalk and Wire provides a text box for student reflection. Students can upload 
additional documents to support their reflective comments. 


Connection and linking 
capabilities 


Although Chalk and Wire offers opportunities for providing links to other online 
work, as well as uploading work, limited interaction can be created between and 
among work products. 


Organizational flexibility 


Chalk and Wire offeis limited flexibility for reframing portfolios for a variety of 
applications (e.g., school, job search, etc.). 


Display flexibility and 
transportability 


Student portfolios are created and maintained on the Chalk and Wire system. 
Although the uploads and links could be transported to other e-Portfolio systems, 
students would basically be re-creating their portfolios on the new system. 


Data and information 


Chalk and Wire produces a variety of formatted, printable reports. 


Start-up costs and 
maintenance 


Chalk and Wire limits students’ ability to associate, set-up, and edit course 
profiles. The help system is useful, however, not as robust as many users may 
need. The directions regarding system features and use are clear, however, not as 
comprehensive as may be desired. System customization by different departments 
for unique purposes appears limited. 


System functionality and 
usability 


Although it is relatively easy to logon to Chalk and Wire, logging off and 
determining which system’s options are available for use are not readily apparent. 
Selecting user profiles and types, and determining system security features, also 
are not easily identified or applied. However, students can easily create, organize, 
and upload work online, as well as submit assignments online. Also, instructors 
can easily create assignments online, providing corresponding instructions and 
assignment details. 



ePortfolio Resources 

This final section of the paper contains web addresses and brief descriptions of some of the best Internet 
resources on the subject of electronic portfolios. Links to publications and organizations are included. 

ElectronicPortfolios.org at http://www.electronicportfolios.org/- Compiled by Dr. Helen Barrett, an expert 
on electronic portfolios, this web site offers links to presentations, papers, and how-to guides to develop electronic 
portfolios using generic tools. Of particular interest is the link to the Apple Learning Interchange exhibit at 
http ://ali . apple .com/ ali sites/ali/exhibits/ 1000156/ . 

National Learning Infrastructure Initiative at 

http://www.educause.edU/nlii/kevthemes/eportfolios.asp#resources identifies electronic portfolios as a key theme. 
Their web page links to a variety of references and resources on the topic, including conference presentations. 

ePortConsortium.org at http://www.eportconsortium.org/ - A white paper published in November, 2003, by 
this consortium of institutions is downloadable from their site. Additionally, the References page of their web site 
links to a variety of eportfolio projects, software, papers, and activities. 
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Electronic Portfolio Action Committee (EPAC) Virtual Community of Practice (VCoP) at 
https://worktools.si.umich.edu/workspaces/dcamrid/002.nsf - The EPAC VCoP offers a forum for scholars, thinkers, 
and innovators to discuss electronic portfolios in higher education. The American Association of Higher Education 
(AAHE) sponsors this VCoP. Organizational membership is required. 

Also from the AAHE, the Portfolio Clearinghouse at http://www.aahe.org/teaching/portfolio db.htm 
provides a searchable database of electronic portfolio projects from a global audience. 

Sponsored by the Carnegie Foundation, the Knowledge Media Laboratory at 
http://kml2.carnegiefoundation.org/html/gallery.php offers 25 examples of teaching portfolios created by K- 12 
teachers and those in higher education. Examples come from a wide variety of disciplines. 

Summary 

Eportfolio implementers face a number of important decisions. The system selected impacts both the 
pedagogical and technological environments at the institution. Carney’s (2002) dilemmas illustrate the difficulty of 
the decision-making process for instructional leaders. Shulman ( 1998) also warns about five dangers of using 
portfolios: they become mere exhibitions; they are difficult to complete; the consequences of trivialization; the 
possible perversion of the process; and the possibility of misrepresentation. Instructional leaders must guard against 
these dangers and continue to advocate for the pedagogical goals of electronic portfolios or risk being overtaken by 
high-stakes assessment and accreditation goals of administrators, accreditation organizations, and evaluators. 
Furthermore, the trend to aggregate data carries ethical considerations and offers potential for abuse that must be 
weighed carefully. 

Technology leaders have an equally imposing set of challenges to face. The lack of defined standards in the 
industry pose special challenges for ensuring scalability and integration with legacy systems. The Open Source 
movement offers a viable, if non-traditional, option for institutions with substantial in-house expertise. 

A number of customized systems are available, at a price, to facilitate a non-technical user in creating and 
assessing an electronic portfolio. The electronic portfolio industry is still in a nascent stage, so one can expect a 
number of significant changes to both commercial and non-commercial offerings. It is essential curricular, 
pedagogical, and administrative leaders meet with software developers to educate them about the how eportfolio 
systems should be developed. The OSPI and EPAC communities encourage scholars to become involved and guide 
their development efforts. Curriculum and pedagogy experts must involve themselves in this discussion to influence 
eportfolio development efforts. 
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